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1.  Summary 


Computer-generated  display  is  widely  used  these  days  and  decision-making  relies  on  both 
accurate  information  and  efficient  visualization  techniques.  Tactical  symbols  are  often  used  in  the  display 
of  battlefield  information.  In  the  past,  uncertainty  information  associated  with  tactical  symbols  was 
ignored.  Recent  research  explored  uncertainty  display  techniques  and  investigated  the  capabilities  of  the 
human  perceptual  system.  However,  due  to  the  lack  of  an  efficient  tool,  previous  work  focused  only  on 
the  static  display,  and  limited  techniques  have  been  studied.  The  purpose  of  this  project  is  to  develop  a 
software  toolkit,  which  displays  tactical  symbols  dynamically,  and  to  use  it  to  understand  the  influence  of 
visualization  techniques  on  human  decision-making.  This  report  describes  the  design  and  development  of 
the  software  toolkit,  the  experimental  procedure  and  results,  and  the  development  of  a  new  method  for 
dynamic  display  of  uncertainty  -  an  automatic  feature  based  morphing.  Section  2  is  an  introduction  of  this 
project.  Section  3  provides  a  literature  review  of  graphical  display  of  uncertainty  and  previous  human 
factors  study  on  uncertainty  representations.  Section  4  describes  the  functionality  of  the  software  toolkit 
and  its  development.  Section  5  discusses  the  findings  from  the  experiment  conducted  using  the  developed 
toolkit.  Three  uncertainty  display  techniques  (rectangular  bar,  color  saturation  and  blurred  image)  have 
been  compared  and  the  results  support  our  hypothesis.  A  feature  based  morphing  algorithm  is  developed 
and  implemented  in  the  toolkit  for  future  investigation  of  its  capability  of  display  of  uncertainty  for 
tactical  symbols.  Section  6  explains  the  algorithm.  Finally,  we  draw  conclusions  in  Section  7. 


1 


2.  Introduction 


The  uncertainty  associated  with  tactical  symbols  seems  limitless.  There  are  various  uncertainty 
definitions  and  there  is  no  real  consensus.  Here  we  refer  uncertainty  to  a  lack  of  sureness  or  definite 
knowledge  about  an  outcome  result  (Andre  &  Cutler,  1998).  Uncertainty  information  can  be  provided  in  a 
frequency  format  or  a  probability  format,  which  can  be  given  in  linguistic,  numerical,  or  graphical 
representation.  In  our  study,  we  focus  on  the  graphical  display  and  situations  which  require  decisions 
to  be  made  in  a  timely  manner.  A  software  toolkit  was  developed  to  assist  this  study. 


2.1,  Tactical  Symbols 

A  wide  range  of  symbolic  representations  are  used  in  information  visualization,  such  as  Chemoff 
faces  and  multidimensional  icons  (Spence,  2001).  In  the  military,  to  ensure  the  compatibility  and  the 
interoperability  of  DoD  Command,  Control,  Communications,  Computer  and  Intelligence  system 
development,  MIL-STD-2525B  (DoD,  1999)  was  published.  MIL-STD-2525B  is  the  current  standard  for 
war  fighting  symbology.  The  MIL-STD-2525B  defines  two  categories  of  warfighting  symbology:  tactical 
symbols  and  tactical  graphics.  Tactical  symbols  are  point  objects  that  present  information  that  can  be 
pinpointed  in  one  location  at  a  particular  point  in  time.  A  tactical  symbol  is  composed  of  a  frame,  fill,  and 
icon.  At  the  same  time,  it  may  include  text  and/or  graphic  modifiers  that  provide  additional  information. 

•  The  frame  is  the  geometric  border  of  a  symbol,  which  provides  an  indication  of  the  affiliation, 
battle  dimension,  and  status  of  a  warfighting  object. 

•  The  fill  is  the  interior  area  within  a  symbol.  If  a  color  fill  is  used  in  a  framed  symbol,  it  provides 
redundant  information  about  the  affiliation  of  the  object. 

•  The  icon  is  the  innermost  part  of  a  symbol,  which  provides  an  abstract  pictorial  or  alphanumeric 
representation  of  a  warfighting  object.  The  icon  portrays  the  role  or  mission  performed  by  the  object. 

•  A  modifier  provides  optional  additional  information  about  a  symbol. 

We  use  the  fixed  wing  aircraft  symbols  in  our  experiment.  The  friendly  and  hostile  symbols  are 
shown  in  Figure  2-1.  Different  symbols  can  also  be  used  in  the  toolkit  with  their  image  saved  in  portable 
network  format  (PNG)  formats. 


Figure  2-1.  Friendly  and  hostile  s; 


$  used  in  the  experiment. 


2.2,  Technical  Approaches 

This  study  consists  of  several  tasks.  The  goal  and  approach  of  each  task  are  explained  below, 
a)  Literature  review.  The  goal  of  this  task  is  to  review  the  display  methods  that  are  suitable  for 
dynamical  uncertainty  display  of  tactical  symbols  and  to  select  the  methods  for  the  comparison 
experiment.  This  review  was  conducted  by  reviewing  papers  and  reports  in  uncertainty 
visualization  in  broad  fields  and  human  factors  study  on  uncertainty  representation. 
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b)  Design  of  experiment.  The  goal  is  to  design  the  experiment  to  compare  selected  display  methods. 
Scenarios  were  created  to  simulate  real-life  situations  and  to  provide  contexts  of  spatial  and 
state  problems. 

c)  Development  of  a  software  toolkit.  The  goal  of  this  task  is  to  provide  a  tool  for  conducting  the 
experiment.  A  software  toolkit  was  developed  to  display  tactical  symbols  in  designed  scenarios 
with  selected  display  methods  and  to  collect  experimental  data. 

d)  Conducting  the  experiment  and  analyzing  experiment  results.  The  goal  of  this  task  is  to  find  out  if 
there  are  differences  between  the  selected  methods  on  perception  of  uncertainty  and  decision 
making.  The  subjects  were  recruited  from  students  in  the  college  of  engineering  at  North  Carolina 
A&T  State  University.  Data  were  collected  using  the  developed  toolkit  and  analyzed  with 
statistics  software. 

e)  Exploring  new  methods  for  uncertainty  display.  The  goal  is  to  provide  a  method  to  display 
uncertainty  by  shapes.  A  morphing  algorithm  was  developed  and  implemented  in  the  software 
toolkit. 
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3.  Literature  Review 


In  this  literature  review,  information  is  collected  to  allow  designers  to  determine  what  issues  are 
important  when  designing  the  visualization  tool  for  military  applications,  to  select  methods  for  display 
and  visualization,  and  to  understand  how  humans  detect  uncertainty  changes.  Section  3.1  reviews 
uncertainty  visualization  techniques.  Section  3.2  provides  a  review  on  dynamic  display  techniques. 
Section  3.3  discusses  previous  studies  on  the  impact  of  uncertainty  visualization  techniques  on  decision 
making.  Section  3.4  lists  existing  toolkits. 


3.1.  Uncertainty  Visualization 

Uncertainty  information  can  be  provided  in  a  frequency  format  or  a  probability  format,  which  can 
be  given  in  linguistic,  numerical,  or  graphical  representation.  Kirschebaum  and  Arruda  (1994)  studied 
effects  of  graphic  and  verbal  probability  information  on  command  decision  making  within  a  context  of  a 
spatial  problem.  Their  results  suggest  a  graphic  representation  of  uncertainty  may  considerably  improve 
the  judgments  of  decision  makers.  Bisantz,  Marsiglio,  and  Much  (2005)  investigated  different  uncertainty 
representation  formats’  influence  on  decisions  of  stock  purchasing.  The  representation  formats  include 
range,  numeric,  linguistic,  colored  icon,  and  arrow  icon.  The  information  was  updated  every  20  seconds. 
Their  results  showed  there  were  no  main  effects  of  display  format. 

Visualization  of  uncertainty  has  been  investigated  in  many  fields.  Foody  and  Atkinson  (2002) 
gave  an  overview  of  the  most  recent  studies  in  modeling  the  uncertainty  in  Geographic  Information 
Systems.  Botchen,  Weiskopt,  and  Ertl  (2005)  discussed  texture-based  visualization  of  uncertainty  in  flow 
fields.  Brown  (2004)  analyzed  present  visual  features  used  to  indicate  uncertainty  and  summarized  as 
follows: 

•  Intrinsic  representation  -  position,  size,  brightness,  texture,  color,  orientation  and  shape 

•  Further  related  representations  -  boundary  (thickness,  texture  and  color),  blur,  transparency 
and  extra  dimensionality 

•  Extrinsic  representation  -  dial,  thermometers,  arrows,  bars,  different  shapes  and  complex 
objects  -  pie  charts,  graphs,  complex  error  bars 

Many  visualization  techniques  have  been  applied  to  these  features  in  uncertainty  visualization. 
Andre  and  Cutler  (1998)  used  rings  to  represent  uncertainty  regarding  the  location  of  an  entity. 
Kirschenbaum  and  Arruda  (1994)  used  an  ellipse-shaped  representation  of  the  confidence  interval  of  the 
submarine’s  location.  Finger  and  Bisantz  (2002)  investigated  using  blended  and  degraded  icons.  Pang, 
Wittenbrink,  and  Eodha  (1997)  suggested  modifying  attributes  of  scene  geometry,  such  as  color,  shading, 
reflectivity,  and  bumpiness,  to  indicate  uncertainty.  Wittenbrink,  Pang,  and  Eodha  (1996)  also 
investigated  using  glyphs  (graphical  forms  such  as  arrows  and  vertical  lines)  to  display  data  uncertainty. 
MacEachren  and  DiBiase  (1991)  used  the  traditional  variables  in  cartography  (location,  size,  value, 
texture,  color,  orientation,  and  shape)  along  with  color  saturation  to  display  different  types  of  uncertainty. 
They  also  suggested  representing  uncertainty  using  pairs  of  graphics.  Animation  can  also  be  used  (Brown, 
2004).  The  degree  of  uncertainty  corresponds  to  the  degree  of  motion. 

Pang,  Wittenbrink,  and  Eodha  (1997)  surveyed  techniques  for  presenting  data  together  with 
uncertainty  and  focused  on  scientific  visualization.  Classifications  of  methods  are  reviewed  in  their  paper 
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and  a  classification,  which  accounts  for  different  types  of  uncertainty  information  or  techniques,  is 
proposed.  The  classification  has  five  characteristics:  value,  location,  data  extent,  visualization  extent,  and 
axes  mapping.  They  present  their  results  for  environmental  visualization,  surface  interpolation,  global 
illumination  with  radiosity,  flow  visualization,  and  figure  animation.  The  techniques  include  adding 
glyphs,  adding  geometry,  modifying  geometry,  modifying  attributes,  animation,  sonification,  and  psycho¬ 
visual  approaches.  A  common  classification  method  for  scientific  visualization  is  based  on  data 
dimension.  The  algorithms  can  be  categorized  into  scalar,  vector,  and  tensor  algorithms.  Unlike  scientific 
visualization,  visualization  of  tactical  symbols  does  not  have  a  natural  geometric  structure.  Most 
techniques  developed  for  uncertainty  visualization  in  volume  rendering  are  not  suitable  for  tactical 
symbol  display.  In  this  section,  we  will  review  the  techniques  in  the  following  categories:  applying  simple 
geometry,  modifying  geometric  attributes,  modifying  geometry  and  animation.  We  then  discuss  the 
possible  applications  to  tactical  symbol  display. 

3.1.1.  Adding  Simple  Geometry 

In  this  category,  simple  graphical  shapes  (line,  arrow,  arc,  ring,  ellipse  etc)  are  used.  These  shapes 
are  easy  to  create.  They  might  be  suitable  for  displaying  uncertainty  in  different  situations.  Glyphs  are 
usually  used  for  vectors  in  scientific  visualization  where  the  data  are  physically  based.  For  example, 
Wittenbrink,  Pang,  and  Lodha  (1996)  used  arrows  to  represent  magnitude  and  direction  of  winds  and 
ocean  currents  along  with  the  uncertainties  in  these  dimensions.  The  general  shape  of  the  glyph  was  a  line 
and  arrow;  the  width  of  the  arrowhead  represented  uncertainty  in  heading  and  multiple  arrowheads 
represented  uncertainty  in  magnitude.  In  another  example,  the  area  of  the  glyph  was  used  to  represent 
magnitude.  Parameters  of  shapes  such  as  an  arc,  an  ellipse,  and  a  ring  might  be  easily  related  to  the 
uncertainty.  For  example,  the  radius  of  the  circle  is  used  to  signal  the  position  uncertainty  in  investigation 
of  uncertainty  display  in  aviation  navigation  systems  (Andre  &  Cutler,  1998).  Arcs  are  utilized  to  display 
heading  uncertainty.  The  missing  segment  of  a  ring  represents  the  uncertainty  region.  In  another  example, 
a  study  of  submarine  sonar  display  by  Kirschenbaum  and  Arruda  ( 1 994)  provided  a  graphical  display  of 
the  area  of  uncertainty  regarding  the  location  of  the  target.  The  target  icon  is  surrounded  with  an 
uncertainty  ellipse.  The  center  of  the  ellipse  is  at  the  target  location  and  the  shape  gives  the  range  and 
bearing  probability  distribution. 

3.1.2.  Modifying  Geometric  Attributes 

Geometric  attributes  can  be  used  to  represent  uncertainty.  However,  representing  uncertainty 
prevents  these  graphical  dimensions  from  being  utilized  for  other  purpose.  Attributes  include  color, 
shading,  surface  normal,  lighting,  and  texture.  The  simplest  approach  is  to  use  a  color  lookup  table  where 
a  color  palette  is  used  to  map  uncertainty  values  to  different  colors.  Hengl  (2003)  discussed  using  Hue- 
Saturation-Intensity  (HSI)  color  models  in  visualization  of  uncertainty  for  continuous  interpolated  and 
categorical  data.  Whiteness  or  paleness  is  used  as  a  variable  to  display  uncertainty.  Color  legends  are 
given  in  two-dimensional  legends  or  color  wheels.  In  two-dimensional  legends,  hues  change  on  the 
vertical  axis,  and  saturation  and  intensity  change  linearly  from  low  to  total  whiteness.  In  color  wheels, 
hues  change  on  the  perimeter,  and  radial  distance  represents  the  confusion  (whiteness).  Then  the 
predictions  and  prediction  error  for  continuous  variables  can  be  visualized  by  mapping  predictions  to  hues 
and  errors  to  whiteness.  For  categorical  data,  class  hues  are  selected  first  in  the  color  wheel  reflecting  the 
taxonomic  similarity  of  the  classes.  Then  the  hues,  saturations,  and  intensities  are  mapped  to  Red,  Green, 
and  Blue  (RGB)  values.  The  color  at  each  pixel  is  an  averaged  RGB  value  of  multiple  memberships  at 
that  point. 
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Uncertainty  values  can  also  be  mapped  to  reflectivity  coefficients  in  shading  such  as  specular  and 
diffuse.  Pang,  Wittenbrink,  and  Lodha  (1997)  showed  the  different  effects  of  representing  uncertainty  by 
altering  diffuse  coefficients,  altering  specular  coefficients  and  mapping  different  values  to  2D  and  3D 
textures.  Examples  of  manipulating  surface  normals  can  be  found  in  their  paper.  Hootsmans  (2002) 
compared  visualization  of  uncertainty  using  different  color  variables  -  saturation,  intensity  and  hues. 
Jiang  (1996)  investigated  some  new  fuzzy  color  systems  for  visualization  of  uncertainty. 

3.1.3.  Modifying  Geometry 

Uncertainty  can  also  be  visualized  by  modifying  geometry.  Geometry  can  be  transformed, 
subdivided,  or  refined.  For  example,  uncertainty  can  be  mapped  to  the  translation  or  rotation  of 
primitives,  or  related  to  how  the  geometry  is  warped  or  distorted.  We  also  include  blurred  images  in  this 
category.  Blurring  is  removal  of  spatial  high  frequency  details  from  information  which  reduces  the 
viewer’s  ability  to  recognize  fine  features  (Russ,  1992). 

Finger  and  Bisantz  (2002)  pointed  out  that  blurring  or  fuzziness  corresponds  to  a  natural  way  of 
conveying  the  uncertainty.  They  performed  experiments,  which  compared  the  use  of  numeric  formats  to 
degraded  and  blended  icons  for  conveying  situational  estimates.  An  image  can  be  blurred  by  using 
different  types  of  filters. 

The  visual  effect  of  a  Gaussian  blur  resembles  viewing  the  images  through  an  out-of- focus  lens. 
A  normal  distribution  is  used  for  calculating  the  transformation  to  apply  to  each  pixel  in  the  image.  Each 
pixel  in  the  blurred  image  is  an  average  of  the  original  pixel’s  value  and  its  neighboring  pixels.  According 
to  the  distribution,  neighboring  pixels  receive  smaller  weights  as  their  distance  to  the  original  pixel 
increases.  The  pixelized  filter  takes  a  square  area  and  gives  it  the  mean  color  value  of  the  pixels  it 
contains,  while  the  motion  blur  filter  averages  frames. 

3.1.4.  Animation 

Animation  parameters  such  as  speed  or  duration,  motion  blur,  range  or  extent  of  motion  can  be 
mapped  to  uncertainty  values.  Brown  (2004)  investigated  animated  visual  vibrations  as  an  uncertainty 
visualization  technique.  Two  values  are  allowed  to  exist  in  the  same  spatial  location.  An  animation 
vibrates  between  the  two  values  to  indicate  uncertainty.  The  feature  is  a  function  of  time,  period,  floor 
value,  and  ceiling  value.  The  oscillation  function  determines  the  nature  of  the  transitions.  Three  functions 
are  used:  step,  linear,  and  sinusoidal  in  Brown’s  study. 

3.2.  Dynamic  Display  Techniques 

When  updating  the  information  on  a  tactical  map,  sudden  changes  of  symbols  might  cause 
confusion  and  unpleasant  visual  effects.  Users  might  also  miss  capturing  the  changing  of  status. 
Morphing  is  an  image  processing  technique  used  for  the  metamorphosis  from  one  image  to  another.  The 
idea  is  to  get  a  sequence  of  intermediate  images,  which  represent  the  change  from  one  image  to  the  other. 

Costa,  Darsa,  Velho,  and  Wolberg  (1995)  summarized  common  morphing  techniques  in  their 
lecture  notes  for  Special  Interest  Group  on  GRAPHics  and  Interactive  Techniques  (SIGGRAPH)  95. 
Cross  dissolve  is  a  technique,  which  has  been  used  by  the  film  industry  for  decades.  The  source  image 
and  the  destination  image  are  superimposed  and  their  color  values  are  blended.  The  values  are 
continuously  changed  from  100%  of  the  source  image  to  100%  of  the  destination  image,  and  a  smooth 
transition  is  achieved.  It  is  usually  used  to  attain  metamorphosis  effect  between  two  different  objects  with 
similar  shapes.  Cross-dissolve  does  not  involve  geometry  alignment.  It  is  not  so  effective  in  suggesting 
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the  actual  metamorphosis.  For  morphs  between  faces,  the  metamorphosis  does  not  look  good  if  the  two 
faces  do  not  have  the  same  shape  approximately. 

To  consider  both  the  shape  and  attribute,  warping  between  the  source  and  destination  images  are 
required.  Three  specification  techniques  that  have  been  largely  used  are  specification  by  partition, 
specification  by  features,  and  automatic  specification.  Specification  by  partition  is  also  called 
specification  by  meshes.  Two  meshes,  with  equivalent  combinatorial  topologies,  are  created  as  part  of  the 
geometric  data  set  of  the  graphical  object.  Each  mesh  defines  a  partition  of  the  object  domain.  The 
transformation  between  two  corresponding  meshes  is  specified  to  attain  the  desired  results.  For  example, 
the  points  on  the  source  mesh  will  move  to  the  destination  mesh,  while  the  neighboring  points’  movement 
will  be  determined  by  a  weighting  function,  which  depends  on  the  relative  distance  from  the  point  to  each 
mesh  point.  Radial  projection  is  a  strategy  of  this  type.  It  is  often  used  for  transforming  star-shaped 
polygons.  The  points  on  the  curve  are  projected  to  a  circle  and  a  linear  path  is  taken  to  morph  to  another 
curve  (Pomm  &  Werlen,  2004).  Different  than  specification  by  partition  which  is  on  the  entire  domain, 
feature  based  specification  only  distinguishes  features.  Feature  transformations  are  specified  by  users. 
Each  feature  can  be  described  by  a  point  or  a  curve.  For  example,  if  each  feature  is  described  by  a  line, 
morphing  between  two  images  can  be  done  by  first  defining  corresponding  features  in  the  source  and 
destination  images,  and  then  mapping  between  the  lines  specified.  Depending  on  the  intermediate  frames 
required,  a  set  of  interpolated  lines  are  obtained.  An  intermediate  frame  is  obtained  by  warping  the  source 
lines  to  the  intermediate  lines,  warping  the  destination  lines  to  the  intermediate  lines,  and  then  combining 
the  warped  images  proportionally  depending  on  how  close  the  frame  is  with  respect  to  the  initial  and  final 
frames. 

The  warping  specification  techniques  described  above  enable  the  user  to  specify  the 
transformation  only  at  some  finite  number  of  elements  belonging  to  the  geometric  data  set  of  the 
graphical  object.  Automatic  specifications  are  detected  by  some  automatic  algorithm.  However,  despite 
some  attempts,  use  of  the  automatic  specifications  is  still  rare. 


3.3.  Impact  of  Uncertainty  Visualization  on  Decision  Making 

Prior  research  has  investigated  the  effects  of  graphical  representation  of  uncertainty. 
Kirschenbaum  and  Arruda  (2004)  studied  the  performance  effects  of  graphic  and  verbal  representations  of 
uncertainty  within  the  context  of  a  spatial  problem.  Their  results  suggested  a  graphic/spatial 
representation  of  uncertainty  may  considerably  improve  the  judgments  of  decision  makers.  Andre  and 
Cutler  (1998)  compared  a  numeric,  an  arc  and  a  ring  representation  of  uncertainty  regarding  an  object’s 
heading  in  a  simulated  anti-aircraft  task.  The  arc  representation  provided  a  slight  advantage  over  the  other 
two  methods. 

Bisantz,  Marsiglio,  and  Much  (2005)  conducted  studies  on  probabilistic  information 
representation.  Participants  were  asked  to  generate  membership  functions  corresponding  to  three 
uncertainty  display  formats.  They  found  a  high  degree  of  similarity  in  functions  across  formats  and 
participants  and  a  strong  relationship  between  the  shape  of  the  membership  function  and  the  intended 
meaning  of  the  representation.  In  another  three  experiments  on  simulated  stock  purchase  task,  information 
of  stock  profitability  was  probabilistic.  Their  results  showed  few  performance  differences  attributable  to 
display  formats  (blurred  and  colored  icons,  linguistic  phrases,  numeric  expressions,  and  combinations). 
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In  similar  studies  on  utilizing  graphical  formats  to  convey  uncertainty,  Finger  and  Bisantz  (2002) 
conducted  a  classification  study,  which  showed  that  participants  could  sort,  order,  and  rank  icons  from 
five  sets  intended  to  present  different  levels  of  uncertainty.  Then,  three  icon  sets  were  selected  for  an 
experiment  in  which  participants  had  to  identify  the  status  of  contacts  as  either  hostile  or  friendly. 
Contacts  and  probabilistic  estimates  of  their  identities  were  depicted  on  a  simulated  radar  screen  in  one  of 
three  ways:  with  degraded  icons  and  probabilities,  with  non-degraded  icons  and  probabilities,  and  with 
degraded  icons  only.  Their  results  showed  that  participants  using  displays  with  only  degraded  icons 
performed  better  on  some  measures  than  the  other  tested  conditions. 


3.4.  Toolkits 

McQueary,  Krause,  Santos,  Wang,  and  Zhao  (2004)  described  the  design  of  the  Uncertainty 
Prediction  System  (UPSYS),  which  supports  the  analysis  and  visualization  of  battlefield  uncertainty 
calculation  within  the  context  of  predictive  battlespace  awareness.  Mewett  and  Clark  (2005)  introduced 
the  development  of  the  Intelligence  Surveillance  and  Reconnaissance  Dynamic  Display  (ISRDD). 
Slocum,  Clibum,  Feddema,  and  Miller  (2003)  described  the  development  of  software  that  is  intended  to 
enable  decision  makers  (and  their  scientific  advisors)  to  visualize  uncertainties  associated  with  the  future 
global  water  balance.  Brown  (2004)  implemented  animated  visual  vibrations  for  uncertainty  visualization 
in  the  Framework  for  Experimental  Visualization  in  Education  and  Research  (FEVER),  which  is  an 
extension  to  OpenSceneGraph.  However,  no  tools  have  been  developed  to  assist  human  factors  study  of 
uncertainty  visualization. 


Uncertainty  display  techniques  have  been  reviewed  above  in  four  categories:  applying  simple 
geometry,  modifying  geometric  attributes,  modifying  geometry,  and  animation.  Three  techniques  are 
selected  from  among  three  categories  to  implement  in  the  toolkit:  1)  Applying  simple  geometry  —  a 
rectangular  bar  is  added  at  the  top  of  the  symbol  to  show  the  probability  level;  2)  Modifying  geometric 
attributes  —  the  color  is  the  attribute  to  be  modified,  and  whiteness  (or  yellowness)  is  used  as  a  variable  to 
display  uncertainty;  3)  Modifying  geometry  —  blurred  images  will  be  used.  The  three  methods  are 
implemented  in  the  toolkit  to  realize  dynamic  display:  sudden  change,  cross-dissolve,  and  feature-based 
morphing. 
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4.  Software  Toolkit  Development 


A  software  toolkit  was  developed  to  earry  out  the  study.  The  goal  of  the  toolkit  is  to  provide  a 
tool  to  display  taetieal  symbols  dynamieally,  to  ereate  the  designed  seenarios  and  to  obtain  experimental 
data.  The  toolkit  is  developed  in  C++  using  Qt  library.  The  main  window  eontains  a  menu  bar,  a  tool  bar, 
two  doek  widgets,  a  status  bar,  and  a  display  area  (Figure  4-1).  By  default,  the  left  doek  widget  is  the 
seenario  setting  dialog  box,  and  the  right  dock  widget  is  the  user  input  dialog  box.  The  toolbar  contains 
two  buttons  (play  and  stop)  for  playing  the  selected  scenario. 


Figure  4-1.  Screenshot  of  the  user  interface. 

The  program  runs  in  two  modes.  Under  the  experimenter  mode,  data  will  not  be  saved  before  a 
new  scenario  is  played.  Experimenters  can  focus  on  adjusting  the  scenario  parameters,  which  can  be 
loaded  from  parameter  files.  Appendix  A  contains  the  user  manual  of  the  toolkit.  The  parameter  file 
format  is  described  in  the  manual.  While  under  the  subject  mode,  data  will  be  forced  to  save  before  a  new 
scenario  is  played  to  guarantee  collecting  all  the  data  in  experiments. 


4.1.  Use  Case  and  Class  Diagram 

The  toolkit  is  developed  using  Object-oriented  Programming.  Figure  4-2  is  a  use  case  diagram  of 
the  system.  It  provides  the  functionalities.  Three  actors,  experimenter,  subject,  and  the  toolkit,  interact 
with  the  system.  The  explanation  of  the  use  cases  are  given  below. 
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Figure  4-2.  Use  case  diagram  of  the  system. 


Change  background: 

The  subject  changes  the  background  to  simulated  radar  screen,  white  background  or  user 
defined  images.  The  toolkit  updates  the  background  and  reloads  the  symbols. 

Set  mode: 

The  experimenter  sets  the  mode  to  experimenter  or  subject.  In  the  subject  mode  the  toolkit 
changes  its  setting  so  data  are  saved  before  a  scenario  can  be  played  with  the  exception  for 
the  first  time. 

Display  symbols: 

The  subject  selects  display  symbols  from  the  help  menu.  The  toolkit  displays  the  techniques 
used  and  the  symbols  (both  friendly  and  hostile)  with  probability  from  0  -  1,  in  tenths.  This  is 
used  for  training  purposes.  The  current  scene  will  be  cleared  before  displaying  these  symbols. 
The  displayed  symbols  will  be  removed  from  the  scene  before  a  scenario  is  played. 

Play  scenario: 

The  experimenter  or  subject  plays  the  scenario.  If  it  is  running  under  the  subject  mode,  the 
toolkit  checks  whether  the  data  have  been  saved.  If  not,  it  gives  an  error  message  and  does 
not  play  the  new  scenario. 
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Stop  scenario: 

The  experimenter  stops  playing  the  scenario.  If  the  toolkit  runs  under  the  subject  mode,  the 
stop  button  is  disabled. 

Select  scenario 

The  subject  changes  the  scenario  to  be  played.  The  toolkit  regenerates  the  scenario. 

Select  display  technique: 

The  subject  changes  the  display  technique.  The  toolkit  regenerates  the  scenario. 

Create  scenario: 

The  toolkit  regenerates  the  scenario  once  the  scenario  or  display  method  options  have  been 

reset. 

Load  parameters: 

The  subject  loads  parameters  from  a  text  fde.  The  toolkit  updates  the  parameters’  values. 
Respond  to  questions: 

The  subject  gives  feedback  based  on  scenario.  For  example,  when  scenario  1  is  played, 
subjects  are  requested  to  click  the  radio  button  once  they  detect  changes,  and  recall  the  start 
and  end  probabilities.  The  toolkit  saves  these  data. 

Set  parameters: 

The  experimenter  sets  parameters  in  a  text  file. 

Render  symbols: 

The  toolkit  renders  the  dynamic  symbols  on  the  screen. 


The  class  diagram  of  the  toolkit  is  given  in  Figure  4-3.  The  main  window  contains  four  input  widgets  and 
a  scenario  selection  widget.  These  input  widgets  take  the  responsibility  to  obtain  user  input.  The  scenario 
selection  widget  is  in  charge  of  setting  the  scenarios  and  display  techniques,  and  sending  signals  to  the 
main  window  to  update  user  interface  for  different  scenarios.  The  main  window  also  contains  a  scenario 
creator.  The  scenario  creator  maintains  a  list  of  dynamic  symbols.  The  dynamic  symbol  class  takes  the 
responsibility  of  creating  a  series  of  symbols  by  interpolating  the  start  and  end  symbols. 


4.2.  Symbols  and  Background  Images 

Symbols  can  be  loaded  from  Portable  Network  Graphics  format  (PNG)  formats.  Once  loaded,  the 
image  will  be  scaled  to  33*33  pixels.  The  class  cSymbolImage  takes  the  responsibility  of  loading  images 
and  converting  them  to  pixmaps.  The  toolkit  also  provides  classes  to  create  primitive  shapes  such  as 
circle,  square,  star,  and  pentagon. 

By  default,  the  background  image  is  a  simulated  radar  screen.  However,  the  image  formats, 
including  PNG,  Joint  Photographic  Experts  Group  (JPEG)  or  Bitmap  (BMP)  can  be  loaded  as 
background.  It  can  also  be  set  to  white. 
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4.3,  Creating  a  Dynamic  Symbol 

Each  dynamic  symbol  contains  start  and  end  symbols.  It  creates  a  series  of  images  by 
interpolating  these  two  images.  Options  of  displaying  a  dynamic  symbol  include  sudden  change,  cross¬ 
dissolve,  and  feature-based  morphing.  In  sudden  change,  no  interpolation  is  required.  Start  images  last  for 
a  certain  period  of  time  and  then  the  end  image  replaces  it.  In  cross-dissolve,  images  are  superimposed 
and  their  color  values  are  blended.  The  proportion  of  image  A  and  B  changes  from  1 00%  of  A  and  0%  of 
B  to  0%  of  A  and  100%  of  B.  Figure  4-4  shows  the  effect  of  morphing  from  a  friendly  symbol  to  a  hostile 
symbol.  Feature-based  morphing  is  explained  in  Section  6. 
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Figure  4-3.  Class  diagram  of  the  software  toolkit. 


Figure  4-4.  A  frieudly  symbol  gradually  chauges  to  hostile  by  cross-dissolve. 
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4.4,  Implementation  of  Display  Techniques 

In  section  2,  uncertainty  display  techniques  are  reviewed  and  three  techniques  from  three 
categories  are  selected  to  be  compared.  Figure  4-5  shows  friendly  and  hostile  symbols  displayed  in  the 
toolkit  with  these  techniques. 
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Figure  4-5.  Symbols  displayed  using  three  techniques. 


Rectangular  bar  technique  draws  a  rectangular  bar  above  the  symbol  image.  The  fill  of  the  rectangular 
bar  corresponds  to  the  probability. 

Color  saturation  uses  the  whiteness  of  a  symbol  to  display  the  uncertainty.  A  color  is  represented  in 
hue,  saturation  and,  value  (FISV).  The  saturation  corresponds  to  the  probability.  The  saturation  of  each 
symbol’s  fill  is  set  to  the  product  of  its  original  saturation  value  and  the  probability.  The  image  is  equal  to 
the  original  when  the  probability  is  equal  to  1.  It  has  a  white  fill  when  the  probability  is  equal  to  0. 

Blurred  image  uses  a  pixelized  filter.  It  takes  a  square  area  and  gives  it  the  mean  color  value  of  the 
pixels  it  contains.  The  transparent  neighbors  and  the  neighbors,  which  are  out  of  boundary,  are  simply 
ignored.  The  length  of  the  square  area  corresponds  to  the  probability.  When  the  probability  equals  1 ,  the 
length  is  one  pixel,  thus  the  image  is  the  same  as  the  original  image.  When  the  probability  equals  0,  the 
length  is  ten  pixels.  The  lengths  of  the  area  corresponding  to  other  probabilities  are  interpolated  linearly. 
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5.  Methods 


Research  on  visualization  is  two-fold,  to  investigate  how  to  display  information,  and  to  study  how  the 
information  is  perceived.  An  experiment  was  conducted  to  compare  the  three  display  techniques  selected 
from  different  categories  to  understand  if  there  is  difference  among  them  on  decision-making  in  dynamic 
display,  and  to  provide  guidelines  for  selecting  uncertainty  display  techniques  in  the  future. 


5.1.  Participants 

Twenty  participants,  undergraduate  and  graduate  students  from  the  college  of  engineering  at 
North  Carolina  A&T  State  University,  participated  in  the  experiment  voluntarily.  Eighteen  of  them 
finished  six  visits.  The  average  age  was  23  years,  with  a  standard  deviation  of  6.17.  There  were  four 
female  and  14  male  participants.  Their  computer  game  playing  skills  were  also  evaluated  by  how  many 
years  they  had  played  computer  games.  Participants  were  classified  as  experts  if  they  played  computer 
games  for  more  than  five  years.  Among  the  18  subjects  who  finished  all  the  visits,  six  of  them  were 
novices  and  12  were  experts. 

5.2,  Procedure 

For  the  first  three  visits,  the  scenarios  were  organized  to  sets.  Each  set  contains  scenarios  1  to  5. 
Parameters  in  each  set  were  different.  Participants  ran  seven  sets  of  scenarios  using  the  three  display 
methods.  During  the  first  visit,  they  were  given  training  and  practiced  for  all  scenarios.  They  were  also 
requested  to  sort  the  symbols,  which  were  randomly  disordered,  with  increasing  probability  so  that  they 
were  familiar  with  the  mapping  of  probability.  All  participants  experienced  no  difficulty  sorting  the 
symbols.  In  the  next  three  visits,  participants  ran  scenarios  in  sequence.  For  each  scenario,  the  three 
display  methods  were  used,  and  for  each  display  method,  participants  ran  ten  randomly  set  parameters.  To 
avoid  confounding  of  the  display  methods,  participants  followed  three  orders:  rectangular  bar — color 
saturation — blurred  image,  color  saturation — blurred  image — rectangular  bar,  blurred  image — ^rectangular 
bar — color  saturation.  Each  visit  took  30-45  minutes. 
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6.  Results 


6.1.  Scenario  1 

In  scenario  1 ,  a  tactical  symbol  at  a  certain  location  will  gradually  change  from  one  state  to  the 
other;  for  example,  from  friendly  (with  probability  pi)  to  hostile  (with  probability  p2).  Participants  set 
the  radio  button  to  indicate  when  they  detect  a  change.  The  timer  stops  when  the  radio  button  is  clicked, 
and  the  time  used  to  detect  the  change  is  recorded.  After  the  symbol  stopped  changing,  participants  recall 
the  starting  and  end  states  level  of  uncertainty.  This  scenario  is  to  determine  if  display  formats  affect 
detection  of  change  and  perception  of  uncertainty  in  a  fixed  location.  In  each  trial,  the  location  of  the 
symbol  and  the  start  and  end  probabilities  vary.  The  dynamic  symbol  lasts  for  five  seconds  and  there  are  a 
total  of  ten  frames  between  the  start  and  end  symbols.  Three  frames  of  scenario  1  are  shown  in  Figure  6- 
1. 


Figure  6-1.  Three  frames  from  scenario  1  using  the  rectangular  har  method. 


The  timeliness  and  accuracy  (the  absolute  value  of  user  input  probability  and  the  defined 
probability)  are  analyzed  with  an  Analysis  of  Variance  (ANOVA)  (Table  6-1). 


Table  6-1.  ANOVA  of  Scenario  1 


Dependent  Variable:  time 


Source 

Type  III  Sum 
of  Squares 

Df 

Mean  Square 

F 

Sig. 

Model 

6961.075(a) 

6 

1160.179 

228.893 

.000 

Method 

28.826 

2 

14.413 

2.844 

.059 

skilljevel 

448.404 

1 

448.404 

88.466 

.000 

method  *  skilljevel 

15.776 

2 

7.888 

1.556 

.212 

Error 

2706.655 

534 

5.069 

Total 

9667.730 

540 

a  R  Squared  =  .720  (Adjusted  R  Squared  =  .717) 
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Table  6-1  ANOVA  of  Scenario  1  (Continued) 


Dependent  Variable:  start  probability  error 


Source 

Type  III  Sum 
of  Squares 

Df 

Mean  Square 

F 

Sig. 

Model 

27.587(a) 

6 

4.598 

108.050 

.000 

Method 

3.659 

2 

1.829 

42.990 

.000 

skilljevel 

.297 

1 

.297 

6.972 

.009 

method  *  skilljevel 

.023 

2 

.011 

.265 

.768 

Error 

22.723 

534 

.043 

Total 

50.310 

540 

a  R  Squared  =  .548  (Adjusted  R  Squared  =  .543) 
Dependent  Variable:  end  probability  error 


Source 

Type  III  Sum 
of  Squares 

df 

Mean  Square 

F 

Sig. 

Model 

20.965(a) 

6 

3.494 

75.617 

.000 

Method 

2.257 

2 

1.128 

24.419 

.000 

skilljevel 

.645 

1 

.645 

13.966 

.000 

method  *  skilljevel 

.026 

2 

.013 

.278 

.757 

Error 

24.675 

534 

.046 

Total 

45.640 

540 

a  R  Squared  =  .459  (Adjusted  R  Squared  =  .453) 


There  was  a  significant  effect  of  display  method  on  accuracy,  but  no  significant  effect  on  detection 
time.  On  average,  it  took  about  7  frames  (3.45  seconds)  for  participants  to  detect  the  change.  For  both 
start  and  end  symbols,  the  median  error  of  rectangular  bar  is  the  smallest  (Figure  6-2).  For  start  symbols, 
the  median  errors  of  the  color  saturation  and  blurred  images  are  similar,  and  for  end  symbols,  the  median 
error  of  color  saturation  is  the  greatest.  Rectangular  bar  also  gives  the  smallest  means  and  standard 
deviations  of  errors  (Table  6-2),  while  color  saturation  gives  the  greatest  means.  There  was  significant 
difference  on  detection  time  and  probability  estimation  between  novices  and  experts.  The  experts 
performed  better  with  a  shorter  detection  time  and  higher  accuracy. 

Table  6-2.  Descriptive  Statistics  for  Scenario  1 


a)  Descriptive  statistics  of  probability  estimation  errors  for  the  three  selected  methods 


Method 

N 

Minimum 

Maximum 

Mean 

Std. 

Deviation 

Rectangular  bar 

Start  probability  error 

180 

.00 

1.00 

.0939 

.16785 

End  probability  error 

180 

.00 

1.00 

.1067 

.19364 

Color  saturation 

Start  probability  error 

180 

.00 

1.00 

.2983 

.22633 

End  probability  error 

180 

.00 

1.00 

.2678 

.23270 

Blurred  image 

Start  probability  error 

180 

.00 

1.00 

.2317 

.22210 

End  probability  error 

180 

.00 

1.00 

.1722 

.22350 

16 


b)  Descriptive  statistics  of  time  (sec)  and  probability  estimation  errors  for  the  two  skill  levels 


skilljevel 

N 

Minimum 

Maximum 

Mean 

Std.  Deviation 

Novice 

Time 

180 

.10 

36.60 

4.7522 

3.36735 

Start  probability  error 

180 

.00 

1.00 

.2411 

.24627 

End  probability  error 

180 

.00 

1.00 

.2311 

.27141 

Expert 

Time 

360 

.00 

11.50 

2.8192 

1.40811 

Start  probability  error 

360 

.00 

.90 

.1914 

.20981 

End  probability  error 

360 

.00 

.90 

.1578 

.19662 

Figure  6-2.  Box  plots  of  start  and  end  probability  estimation  errors  of  scenario  1. 


6.2.  Scenario  2 

In  scenario  2,  a  tactical  symbol  moves  along  a  path  defined  by  four  points.  At  each  point,  the 
symbol  lasts  for  two  seconds  and  then  moves  to  the  next  point.  The  four  points  are  randomly  placed  on 
the  simulated  radar  screen  with  the  maximum  distance  between  consecutive  points  set  to  1 00  pixel  length. 
The  state  of  the  symbol  does  not  change,  but  the  probability  changes  at  each  location.  Participants  input 
their  estimation  of  the  travelling  path  by  clicking  four  points  on  the  screen  and  probability  by  using  slider 
bars.  This  scenario  determines  if  dynamic  location  of  symbols  affect  understanding  of  uncertainty.  Figure 
6-3  displays  a  user  input  travelling  path. 
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Figure  6-3.  Screenshot  of  a  user  input  travelling  path  (the  symhol  is  displayed  using  the  color  saturation 

method). 

There  was  a  significant  effect  of  display  method  on  uncertainty  estimation  error  (Table  6-3)  but 
no  significant  effect  on  location  estimation  error.  All  p  -  values  identified  as  sig.  In  the  tables  were 
greater  than  .06.  The  uncertainty  estimation  error  is  defined  as  the  absolute  value  of  user  input  probability 
and  the  defined  probability,  and  the  location  estimation  error  is  the  distance  between  the  user-specified 
location  to  the  defined  location.  Rectangular  bar  again  gives  the  smallest  mean.  The  errors  of  the  middle 
points  (2'“*  and  3'^'’  points)  are  slightly  greater  than  the  first  and  last  points  for  rectangular  bar  and  color 
saturation,  but  this  pattern  does  not  show  in  blurred  image  (Table  6-4a).  Subject’s  game  playing  skills  do 
not  affect  the  location  estimation  errors  (all  sig.  values  greater  than  .08),  but  playing  skills  has  a 
significant  effect  on  the  uncertainty  estimation  error  for  all  points,  excluding  the  2"'^  point.  Overall  the 
experts  outperform  the  novice  in  uncertainty  estimation  accuracy  (Table  6-4b). 


Table  6-3.  ANOVA  of  Sceuario  2 


Dependent  Variable:  probability  error  at  point  1 


Source 

Type  III  Sum 
of  Squares 

Df 

Mean  Square 

F 

Sig. 

Model 

16.857(a) 

6 

2.809 

68.717 

.000 

Method 

1.904 

2 

.952 

23.280 

.000 

skilljevel 

.174 

1 

.174 

4.255 

.040 

method  *  skilljevel 

.010 

2 

.005 

.119 

.887 

Error 

16.313 

399 

.041 

Total 

33.170 

405 

a  R  Squared  =  .508  (Adjusted  R  Squared  =  .501) 
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Table  6-3  ANOVA  of  Scenario  2  (Continued) 


Dependent  Variable:  probability  error  at  point  2 


Source 

Type  III  Sum 
of  Squares 

Df 

Mean  Square 

F 

Sig. 

Model 

15.976(a) 

6 

2.663 

65.205 

.000 

Method 

1.366 

2 

.683 

16.724 

.000 

skilljevel 

.117 

1 

.117 

2.870 

.091 

method  *  skilljevel 

.006 

2 

.003 

.070 

.932 

Error 

16.294 

399 

.041 

Total 

32.270 

405 

a  R  Squared  =  .495  (Adjusted  R  Squared  =  .487) 


Dependent  Variable:  probability  error  at  point  3 


Source 

Type  III  Sum 
of  Squares 

Df 

Mean  Square 

F 

Sig. 

Model 

18.569(a) 

6 

3.095 

66.856 

.000 

Method 

.929 

2 

.465 

10.035 

.000 

skilljevel 

.235 

1 

.235 

5.079 

.025 

method  *  skilljevel 

.092 

2 

.046 

.996 

.370 

Error 

18.471 

399 

.046 

Total 

37.040 

405 

a  R  Squared  =  .501  (Adjusted  R  Squared  =  .494) 


Dependent  Variable:  probability  error  at  point  4 


Source 

Type  III  Sum 
of  Squares 

Df 

Mean  Square 

F 

Sig. 

Model 

16.533(a) 

6 

2.755 

64.344 

.000 

Method 

2.515 

2 

1.257 

29.362 

.000 

skilljevel 

.603 

1 

.603 

14.074 

.000 

method  *  skilljevel 

.715 

2 

.357 

8.342 

.000 

Error 

17.087 

399 

.043 

Total 

33.620 

405 

a  R  Squared  =  .492  (Adjusted  R  Squared  =  .484) 
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Table  6-4.  Descriptive  Statistics  for  Scenario  2 


a)  Descriptive  statistics  of  probability  estimation  errors  at  four  points  for  the  three  selected 
methods 


Method 

Location 

N 

Minimum 

Maximum 

Mean 

Std.  Deviation 

Rectangular  bar 

point  1 

137 

.00 

.90 

.0854 

.17044 

point  2 

137 

.00 

.70 

.1022 

.18128 

point  3 

137 

.00 

1.00 

.1358 

.20027 

point  4 

137 

.00 

.70 

.0701 

.12797 

Color  saturation 

point  1 

142 

.00 

.80 

.2437 

.20122 

point  2 

142 

.00 

.90 

.2514 

.21495 

point  3 

142 

.00 

1.00 

.2521 

.23781 

point  4 

142 

.00 

1.00 

.2415 

.25327 

Blurred  image 

point  1 

126 

.00 

.90 

.2397 

.23395 

point  2 

126 

.00 

1.00 

.2079 

.20846 

point  3 

126 

.00 

.90 

.2302 

.20756 

point  4 

126 

.00 

1.00 

.2214 

.24020 

b)  Descriptive  statistics  probability  estimation  errors  at  four  points  for  the  two  skill  levels 


skilljevel 

Location 

N 

Minimum 

Maximum 

Mean 

Std.  Deviation 

Novice 

Point  1 

130 

.00 

.90 

.2254 

.23733 

Point  2 

130 

.00 

1.00 

.2169 

.23332 

Point  3 

130 

.00 

1.00 

.2462 

.25401 

Point  4 

130 

.00 

1.00 

.2431 

.28447 

Expert 

Point  1 

275 

.00 

.90 

.1716 

.20217 

Point2 

275 

.00 

.90 

.1735 

.19905 

Point  3 

275 

.00 

.90 

.1869 

.20247 

Point  4 

275 

.00 

1.00 

.1462 

.18741 

6.3.  Scenario  3 

In  scenario  3,  a  symbol  gradually  changes  its  probability  at  a  location.  The  symbol  is  enlarged 
and  then  diminished  to  its  original  size.  The  enlarged  size  is  between  1.2  and  1.5  times  of  its  original  size, 
and  the  whole  process  lasts  for  6  seconds  with  a  total  of  21  frames.  Participants  set  the  radio  button  to 
indicate  when  they  detect  a  change.  The  timer  stops  when  the  radio  button  is  clicked  and  the  time  used  to 
detect  the  change  is  recorded.  After  the  symbol  stopped  changing,  participants  recalled  the  start  and  end 
probabilities.  This  scenario  determines  if  display  formats  affect  detection  of  change  and  perception  of 
uncertainty  in  scaling.  In  each  trial,  the  location  of  the  symbol  and  the  start  probabilities,  end 
probabilities,  and  the  enlarged  size  vary.  Cross-dissolve  is  used  to  gradually  morph  from  one  size  to  the 
other. 
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Figure  6-4.  Three  frames  from  scenario  3  using  the  blurred  image  method. 


Table  6-5.  ANOVA  of  Scenario  3 


Dependent  Variable:  time 


Source 

Type  III  Sum 
of  Squares 

df 

Mean  Square 

F 

Sig. 

Model 

5235.754(a) 

6 

872.626 

268.847 

.000 

Method 

26.305 

2 

13.152 

4.052 

.018 

skilljevel 

124.803 

1 

124.803 

38.450 

.000 

method  *  skilljevel 

30.259 

2 

15.129 

4.661 

.010 

Error 

1635.886 

504 

3.246 

Total 

6871.640 

510 

a  R  Squared  =  .762  (Adjusted  R  Squared  =  .759) 


Dependent  Variable:  start  probability  error 


Source 

Type  III  Sum 
of  Squares 

df 

Mean  Square 

F 

Sig. 

Model 

18.371(a) 

6 

3.062 

105.705 

.000 

Method 

.790 

2 

.395 

13.632 

.000 

skilljevel 

.365 

1 

.365 

12.614 

.000 

method  *  skilljevel 

.178 

2 

.089 

3.077 

.047 

Error 

14.599 

504 

.029 

Total 

32.970 

510 

a  R  Squared  =  .557  (Adjusted  R  Squared  =  .552) 


Dependent  Variable:  end  probability  error 


Source 

Type  III  Sum 
of  Squares 

df 

Mean  Square 

F 

Sig. 

Model 

17.518(a) 

6 

2.920 

71.252 

.000 

Method 

.612 

2 

.306 

7.470 

.001 

skilljevel 

.222 

1 

.222 

5.425 

.020 

method  *  skilljevel 

.236 

2 

.118 

2.883 

.057 

Error 

20.652 

504 

.041 

Total 

38.170 

510 

a  R  Squared  =  .459  (Adjusted  R  Squared  =  .453) 
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There  was  a  significant  effect  of  display  method  on  accuracy  and  detection  time  (Table  6-5). 
Rectangular  bar  has  the  shortest  detection  time.  For  both  start  and  end  symbols,  the  median  and  mean 
errors  of  rectangular  bar  are  also  the  smallest  (Table  6-6).  The  mean  errors  of  color  saturation  and  blurred 
image  are  very  close  for  start  symbols.  The  distributions  of  errors  for  start  and  end  symbols  are  also  very 
similar  to  those  of  scenario  1 :  for  start  symbols,  the  medians  are  close  for  color  saturation  and  blurred 
image;  for  end  symbols,  color  saturation’s  median  is  higher  (Figure  6-5).  Skill  levels  affect  both  the 
detection  time  and  accuracy,  and  experts  outperform  novices.  The  difference  on  detection  time  and  start 
probability  was  statistically  significant. 


Table  6-6.  Descriptive  Statistics  for  Scenario  3 


a)  Descriptive  statistics  of  time  (sec)  and  probability  estimation  errors  for  the  three  selected 
methods 


Method 

N 

Minimum 

Maximum 

Mean 

Std.  Deviation 

Rectangular  bar 

Time 

170 

.60 

11.90 

2.9288 

1.61566 

Start  probability  error 

170 

.00 

.90 

.1147 

.17258 

End  probability  error 

170 

.00 

1.00 

.1176 

.20708 

Color  saturation 

Time 

170 

.40 

9.30 

3.1165 

1.94768 

Start  probability  error 

170 

.00 

1.00 

.2112 

.19321 

End  probability  error 

170 

.00 

1.00 

.2118 

.20259 

Blurred  image 

Time 

170 

.40 

10.20 

3.4053 

2.04781 

Start  probability  error 

170 

.00 

.70 

.2171 

.14996 

End  probability  error 

170 

.00 

.90 

.2041 

.20246 

b)  Descriptive  statistics  of  time  (sec)  and  probability  estimation  errors  for  the  two  skill  levels 


skilljevel 

N 

Minimum 

Maximum 

Mean 

Std.  Deviation 

Novice 

Time 

180 

1.10 

9.30 

3.8200 

1.89083 

Start  probability  error 

180 

.00 

1.00 

.2172 

.19486 

End  probability  error 

180 

.00 

1.00 

.2061 

.22724 

Expert 

Time 

330 

.40 

11.90 

2.7848 

1.78287 

Start  probability  error 

330 

.00 

.70 

.1612 

.16634 

End  probability  error 

330 

.00 

1.00 

.1624 

.19550 
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Figure  6-5.  Box  plots  of  start  and  end  probability  estimation  errors  of  scenario  3. 


6.4,  Scenario  4 

In  scenario  4,  a  symbol’s  traveling  path  is  linear.  It  moves  at  a  constant  speed.  When  displayed  to 
subjects,  the  symbols  are  placed  at  a  random  place  around  the  real  location.  The  distance  from  the  real 
location  is  d,  and  its  maximum  value  is  20  pixels,  and  the  probability  is  given  by  1.0-  c//20.  In  each  trial,  a 
different  step  size  and  heading  direction  is  given  and  the  starting  point  is  set  at  a  random  location. 
Participants  are  asked  to  guess  the  next  move  after  observing  several  moves  by  clicking  the  mouse  to 
input  a  point.  We  measured  the  accuracy  of  their  estimations  by  using  the  distance  between  the  real 
location  and  participant’s  input.  This  scenario  is  to  determine  if  display  formats  affect  decision  making  in 
translation.  At  each  point,  the  symbol  lasts  for  3  seconds  and  then  moves  to  the  next  point.  The  affiliation 
of  the  symbol  does  not  change  during  the  process. 


Figure  6-6.  Symbols  displayed  along  its  travelling  patb.  Tbe  line  is  tbe  real  travelling  path,  wbicb  is  not  shown 
to  participants.  Symbols  are  displayed  one  at  a  time. 
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Table  6-7.  ANOVA  of  Scenario  4 


Dependent  Variable:  time 


Source 

Type  III  Sum 
of  Squares 

Df 

Mean  Square 

F 

Sig. 

Model 

29835.635(a) 

6 

4972.606 

267.862 

.000 

Method 

49.458 

2 

24.729 

1.332 

.265 

skilljevel 

362.336 

1 

362.336 

19.518 

.000 

method  *  skilljevel 

3.594 

2 

1.797 

.097 

.908 

Error 

9727.555 

524 

18.564 

Total 

39563.190 

530 

a  R  Squared  =  .754  (Adjusted  R  Squared  =  .751) 


Dependent  Variable:  location  error 


Source 

Type  III  Sum  of 
Squares 

df 

Mean  Square 

F 

Sig. 

Model 

1442467.456(a) 

6 

240411.243 

88.871 

.000 

Method 

2764.125 

2 

1382.063 

.511 

.600 

skilljevel 

7933.482 

1 

7933.482 

2.933 

.087 

group  *  skilljevel 

25199.750 

2 

12599.875 

4.658 

.010 

Error 

1255195.945 

464 

2705.164 

Total 

2697663.401 

470 

a  R  Squared  =  .535  (Adjusted  R  Squared  =  .529) 


Table  6-8.  Descriptive  Statistics  for  Scenario  4 


a)  Descriptive  statistics  of  time  (sec)  and  location  estimation  errors  for  three  selected  methods 


Method 

N 

Minimum 

Maximum 

Mean 

Std.  Deviation 

Rectangular  bar 

Time 

180 

.90 

21.30 

7.0444 

3.69900 

Location  error 

159 

7.62 

271.28 

55.3927 

35.57351 

Color  saturation 

Time 

170 

.90 

24.00 

7.8153 

4.87199 

Location  error 

154 

2.00 

367.02 

54.7378 

61.47173 

Blurred  image 

Time 

180 

.40 

24.40 

7.5106 

4.50530 

Location  error 

157 

3.61 

473.37 

54.1170 

57.20416 

b)  Descriptive  statistics  time  (sec)  and  location  estimation  errors  for  the  two  skill  levels 


skilljevel 

N 

Minimum 

Maximum 

Mean 

Std.  Deviation 

Novice 

Time 

180 

.40 

18.80 

6.3033 

4.15234 

Location  error 

150 

3.61 

473.37 

60.8410 

69.85899 

Expert 

Time 

350 

1.30 

24.40 

8.0397 

4.38223 

Location  error 

320 

2.00 

294.29 

51.8977 

41.65003 

There  was  no  significant  difference  of  time  and  estimation  error  among  the  three  methods  (Table 
6-7).  However,  we  notice  the  different  performance  between  novices  and  experts  (Table  6-8).  Novices 
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tended  to  make  decisions  earlier,  while  experts  would  wait  for  a  longer  time  and  make  decisions  based  on 
more  accurate  information.  The  maximum  distance  to  the  real  location  (associated  with  probability  0)  is 
small  compare  to  the  symbol  size  (33x33  pixels).  The  step  size  is  in  the  range  of  50-100  pixels.  Further 
research  will  investigate  if  changing  the  maximum  distance  will  give  more  insight  and  if  these  methods 
are  adequate  for  displaying  heading  uncertainty. 


6.5.  Scenario  5 

In  scenario  5,  six  symbols,  which  represent  the  target  group,  undergo  a  movement  toward  the 
observer.  When  the  target  is  far  away  from  the  observer,  the  probability  of  getting  accurate  information  is 
low.  As  the  target  moves  toward  the  observer,  the  information  collected  is  more  accurate.  The  probability 
is  a  function  of  the  distance.  P  =  1.0  -  dist  /  MaxDistance ,  where  dist  is  the  current  distance  between 
the  target  and  the  observer,  and  MaxDistance  is  the  distance  at  which  we  assume  the  probability  equals  0. 
The  target  group  can  be  friendly  or  hostile.  A  symbol  will  be  set  by  the  probability.  Assume  that  the  target 
is  friendly.  A  random  number  r  (between  0  and  1)  will  be  generated  for  each  symbol.  If  r  >  P ,  the 
symbol  is  set  as  hostile;  otherwise  it  will  be  set  to  friendly.  (x,y)  is  the  location  of  the  group.  Symbols  are 
located  randomly  in  a  circular  area  centered  at  {x,y)  with  Radius.  The  MaxDistance  and  Radius  can  be  set 
in  the  parameter  file.  Figure  6-7  provides  two  screenshots  of  the  symbols.  On  the  left  figure,  the  target  is 
away  from  the  observer,  while  on  the  right  figure,  it  is  close  to  the  observer.  Cross-dissolve  is  used  for 
changing  state  from  one  to  the  other.  This  scenario  is  to  determine  if  display  formats  improve  change 
detection  of  multiple  moving  target  symbols.  Participants  determine  whether  the  target  is  friendly. 


Figure  6-7.  Two  frames  from  scenario  5  using  the  rectangular  bar  method  (Left:  the  target  is  away  from  the 
observing  point,  right:  the  target  is  close  to  the  observing  point). 


The  time  used  for  participants  to  make  decisions  and  their  choice  (friendly  or  hostile)  were  recorded. 
The  average  probability  of  the  six  symbols  at  the  time  participants  made  choices  is  also  analyzed  using 
ANOVA  (Table  6-9).  There  were  no  significant  differences  on  the  time.  Color  saturation  has  a  smaller 
mean  and  standard  deviation  and  outperforms  rectangular  bar  and  blurred  image  in  accuracy.  The  average 
probability  of  color  saturation  is  the  highest  (Table  6-10).  This  might  be  the  reason  that  participants  made 
more  accurate  choices  when  using  color  saturation  (Figure  6-8).  The  difference  on  time  and  average 
probabilities  of  novices  and  experts  was  not  statistically  significant.  However,  experts  made  correct 
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decisions  at  a  higher  percentage.  Experts  were  correct  60%  of  the  time,  while  novices  were  correct  51% 
of  the  time  (Figure  6-8). 


Table  6-9.  ANOVA  of  Scenario  5 


Dependent  Variable:  time 


Source 

Type  III  Sum 
of  Squares 

df 

Mean  Square 

F 

Sig. 

Model 

58582.153(a) 

6 

9763.692 

312.563 

.000 

Method 

34.424 

2 

17.212 

.551 

.577 

skilljevel 

7.569 

1 

7.569 

.242 

.623 

method  *  skilljevel 

.440 

2 

.220 

.007 

.993 

Error 

14806.567 

474 

31.237 

Total 

73388.720 

480 

a  R  Squared  =  .798  (Adjusted  R  Squared  =  .796) 


Dependent  Variable:  average  probability 


Source 

Type  III  Sum 
of  Squares 

df 

Mean  Square 

F 

Sig. 

Model 

184.642(a) 

6 

30.774 

2214.796 

.000 

Method 

.609 

2 

.305 

21.923 

.000 

skilljevel 

.000 

1 

.000 

.003 

.959 

method  *  skilljevel 

.016 

2 

.008 

.567 

.568 

Error 

6.586 

474 

.014 

Total 

191.228 

480 

a  R  Squared  =  .966  (Adjusted  R  Squared  =  .965) 


Table  6-10.  Descriptive  Statistics  for  Scenario  5 

a)  Descriptive  statistics  of  time  (sec)  and  average  probability  of  the  six  symbols  at  the  time 
the  decision  was  made  for  three  selected  methods 


Method 

N 

Minimum 

Maximum 

Mean 

Std. 

Deviation 

Rectangular  bar 

Avg.  probability 

160 

.26 

.91 

.5798 

.12606 

Time 

160 

1.10 

24.30 

11.4150 

5.46016 

Color  saturation 

Avg.  probability 

160 

.46 

.92 

.6770 

.08509 

Time 

160 

1.30 

23.00 

10.6694 

4.99556 

Blurred  image 

Avg.  probability 

160 

.27 

.87 

.5995 

.13561 

Time 

160 

1.20 

30.80 

11.0431 

6.19713 
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b)  Descriptive  statistics  of  time  (sec)  and  average  probability  of  the  six  symbols  at  the  time 
the  decision  was  made  for  the  two  skill  levels 


skilljevel 

N 

Minimum 

Maximum 

Mean 

Std.  Deviation 

novice 

Time 

120 

1.30 

30.80 

11.2600 

6.04310 

Avg.  Probability 

120 

.27 

.91 

.6193 

.13061 

expert 

Time 

360 

1.10 

27.00 

10.9700 

5.40961 

Avg.  probability 

360 

.26 

.92 

.6186 

.12283 

Figure  6-8.  Frequency  plots  of  the  correct  answers  in  scenario  5. 


6.6.  Workload 

NASA- TLX  (Task  Load  Index)  was  used  to  rate  the  workload.  Subjects  rated  the  workload  after 
running  a  set  of  scenarios  using  one  display  technique.  Within-subject  ANOVA  shows  that  there  were  no 
significant  differences  of  any  of  the  six  factors  and  the  overall  workload  among  the  three  techniques 
(Table  5-11).  Subjects  experienced  high  mental  demand  and  temporal  demand,  and  low  physical  demand 
(Table  5-12). 
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Table  6-11.  ANOVA  of  Workload 


Sum  of 
Squares 

Df 

Mean  Square 

F 

Sig. 

Mental  Demand 

Between  Groups 

17.593 

2 

8.796 

.018 

.982 

Within  Groups 

24597.222 

51 

482.298 

Total 

24614.815 

53 

Physical  Demand 

Between  Groups 

.926 

2 

.463 

.002 

.998 

Within  Groups 

11852.778 

51 

232.407 

Total 

11853.704 

53 

Temporal  Demand 

Between  Groups 

139.815 

2 

69.907 

.129 

.879 

Within  Groups 

27684.722 

51 

542.838 

Total 

27824.537 

53 

Performance 

Between  Groups 

719.444 

2 

359.722 

.692 

.505 

Within  Groups 

26513.889 

51 

519.880 

Total 

27233.333 

53 

Effort 

Between  Groups 

369.444 

2 

184.722 

.519 

.598 

Within  Groups 

18163.889 

51 

356.155 

Total 

18533.333 

53 

Frustration 

Between  Groups 

811.111 

2 

405.556 

.929 

.401 

Within  Groups 

22259.722 

51 

436.465 

Total 

23070.833 

53 

T otal  Workload 

Between  Groups 

159.371 

2 

79.686 

.300 

.742 

Within  Groups 

13563.834 

51 

265.958 

Total 

13723.205 

53 

Table  6-12.  Descriptive  Statistics  for  Workload 


N 

Minimum 

Maximum 

Mean 

Std.  Deviation 

Mental  Demand 

54 

10 

100 

61.85 

21.551 

Physical  Demand 

54 

5 

75 

19.07 

14.955 

Temporal  Demand 

54 

10 

90 

50.09 

22.913 

Performance 

54 

5 

85 

45.56 

22.668 

Effort 

54 

10 

90 

59.44 

18.700 

Frustration 

54 

5 

90 

36.94 

20.864 

Total  Workload 

54 

9.6667 

86.0000 

53.796292 

16.0912524 
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7.  Discussions 


The  three  selected  uncertainty  display  methods’  effects  on  performance  vary  in  different  context.  In 
a  single  symbol  displayed  in  a  fixed  location  with  changing  state  (scenario  1),  rectangular  bar  gives  the 
best  results  on  accuracy.  While  in  multiple  moving  symbol  display  (scenario  5),  color  saturation  gives  the 
most  accurate  results.  This  indicates  that  the  impact  of  uncertainty  display  methods  is  task-related. 
Previous  research  by  Kirschebaum  and  Arruda  (1994)  and  Bisantz  et  al.  (2005)  on  graphical  and  other 
representations  of  uncertainty  also  showed  task-related  results.  Kirschenbaum  studied  graphical  and 
linguistic  representations  for  spatial  uncertainty  and  found  some  differences  between  them,  while  Bisantz 
et.  al."s  study  for  graphical  and  linguistic  representations  for  state  uncertainty  showed  no  differences. 

Both  scenario  1  and  scenario  3  display  one  symbol  at  a  time.  The  symbol  location  is  fixed.  Their 
results  showed  some  similarity.  In  both  scenarios,  rectangular  bar  gave  the  best  results.  The  box  plots  of 
start  and  end  probability  estimation  errors  display  a  similar  pattern. 

The  findings  of  the  study  support  the  hypothesis  that  display  methods  affect  understanding 
uncertainty  and  decision-making  performance  in  dynamic  display.  In  fixed  location  problems  (scenarios  1 
and  3),  the  results  indicate  that  there  is  significant  difference  on  uncertainty  estimation  accuracy  related  to 
the  selected  display  methods,  and  rectangular  bar  gives  the  best  results.  In  translation  problems  (scenarios 
2  and  4),  display  methods  affect  the  perception  of  the  uncertainty  levels,  but  not  the  location  information. 
In  a  problem  with  multiple  symbols  whose  location  and  state  change  continuously  (scenario  5),  color 
saturation  gives  the  best  results.  The  difference  of  workload  of  the  three  methods  is  not  significant. 

In  all  scenarios,  experts  outperform  novices.  Thus  it  indicates  that  understanding  the  dynamically 
displayed  symbols  can  be  improved  by  training  and  practice.  On  the  other  hand,  it  shows  that  none  of  the 
three  selected  display  methods  is  self-explained,  and  their  association  to  the  uncertainty  is  not  intuitive. 

Further  study  needs  to  investigate  the  relation  between  display  formats  and  other  parameters  of 
dynamic  display,  such  as  number  of  frames  and  time  interval.  Future  research  directions  include 
extending  the  comparison  study  for  more  complex  context  and  realistic  battlefield  simulation,  and 
exploring  methods  which  are  self-explained  and  intuitive. 
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8.  Feature-based  Morphing:  A  Potential  Uncertainty 

Display  Method 


Using  shapes  to  represent  uncertainty  is  a  potential  method.  If  two  symbols  correspond  to  probability 
0  and  1,  a  series  of  shapes  morphing  from  one  to  the  other  can  be  used  to  represent  probability  between  0 
and  1.  To  define  the  metamorphosis,  sets  of  correspondence  of  the  two  images  needs  to  be  specified. 
Commonly  used  ways  include  parametric,  partition-based  and  feature-based  specifications.  These 
specifications  require  great  user  interaction.  An  automatic  specification  will  allow  fast  creation  of 
morphing  images.  In  this  chapter,  we  provide  a  morphing  algorithm  based  on  automatic  feature-based 
specification.  Assumptions  have  been  made  that  the  symbols  are  digitized  polygons  with  width  of  one 
pixel.  However,  the  algorithm  can  be  extended  for  more  complex  geometry  in  the  future. 


8.1.  Feature-based  Morphing 

Morphing  is  a  continuous  transition  from  one  graphical  object  to  the  other.  The  process  contains 
specification  and  computation.  Specification  is  an  expression  of  the  desired  results  and  computation  is  a 
way  to  obtain  the  specified  goals  (Berton,  Costa,  Darsa,  Velho,  &  Wolberg,  1995).  Consider  two 
graphical  objects  0^  and  ,  with  geometric  data  sets  [/,  =  Ul  u  U\..LJ\  and  t/j  ~  ^  ^ 

specification  of  a  transformation  between  0^  and  O2  consists  of  a  set  of  ordered  pairs 
?5  =  {(x,,(/,);x,  cz  [/;,(/.  cz[/f} 
and  a  family  of  transformations. 

W'  :  s.  (z  U]  d.  (z  Uf 

The  source  set  S  and  target  set  D  are  defined  as  follows. 

S  =  {x,.  c  I  there  exists  d.  c  sueh  that  {s^,d^)  e  p] 

D  =  {j,.  c  1/2  I  there  exists  c  sueh  that  {s^,di)  e  pj 

Different  configurations  of  S  and  D  result  in  different  specifications.  Specification  by  partition  and 
specification  by  features  have  been  largely  used.  Specification  by  partition  is  also  called  specification  by 
meshes.  Two  meshes  are  created  as  part  of  the  geometric  data  set  of  the  graphical  object.  Each  mesh 
defines  a  partition  of  the  object  domain.  Mapping  of  the  two  meshes  defines  the  mapping  of  the  two 
objects.  Specification  by  features  does  not  specify  the  transformation  for  the  entire  domain,  but  only 
distinguished  features  and  their  transformations  are  specified.  The  specifications  are  relied  on  user  input. 

8.2.  Automatic  Feature-based  Specifications 

We  present  a  method  to  automatically  specify  corresponding  features  without  user  interaction. 
Primitive  geometric  shapes  are  often  used  in  symbols.  These  shapes,  when  digitized,  can  be  considered  as 
polygons.  For  example,  a  circle  can  be  approximated  with  an  inscribed  regular  polygon.  In  many 
applications,  symbols  are  represented  in  digital  images.  If  not,  they  are  first  converted  to  bitmap  images. 
The  feature  lines  are  the  outline  of  the  polygon.  To  map  the  feature  lines  of  two  images,  scan  lines 
starting  from  the  center  of  the  image  are  drawn  to  intersect  the  source  and  target  sets.  Assume  the  feature 
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line  width  is  1  pixel.  The  intersection  of  the  scan  line  and  the  source  and  target  point  sets  form  an  ordered 
pair  inP .  In  continuous  space,  an  intersection  point  can  always  be  obtained  (Figure  8-1).  However,  in 
discrete  space,  intersection  points  are  not  always  found  (Figure  8-2).  To  find  a  map  from  source  to  target 
so  that  every  point  in  the  source  and  target  is  defined  inP ,  we  allow  a  small  change  of  the  slope  of  the 
scan  line  to  find  an  intersection  in  the  neighbor  area. 


Figure  8-1.  Morphing  of  a  star-shaped  polygon  to  a  circle  (Pomm  &  Werlen,  2004). 


(a)  No  intersection  found 

(b)  Intersection  found  by  slightly  decreasing  the  scan  line  slope 

Figure  8-2.  Obtaining  an  intersection  of  the  scan  line  and  a  point  set. 


The  pseudo  code  of  creating  the  ordered  pairs  of  points  is  given  below. 

For  every  point  in  source  point  set 

Create  a  scan  line  starting  from  the  center  and  pass  this  point 
Compute  the  slope  of  the  scan  line 
While  (no  intersection  point  is  found) 

Find  the  intersection  of  the  scan  line  with  the  target  set 
Increase  or  decrease  the  slope  of  the  scan  line 
Save  the  pair  of  point 

Remove  the  target  point  from  the  target  point  set 
For  every  point  remained  in  the  target  point  set 

Create  a  scan  line  starting  from  the  center  and  pass  this  point 
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Compute  the  slope  of  the  scan  line 
While  (no  intersection  point  is  found) 

Find  the  intersection  of  the  scan  line  with  the  source  set 
Increase  or  decrease  the  slope  of  the  scan  line 
Save  the  pair  of  point 


8.3.  Computation 

Once  the  ordered  pairs  of  the  source  and  target  point  sets  are  defined,  the  frames  of  the  morphing 
sequence  are  computed  by  the  linear  interpolation  of  the  source  and  target.  Examples  of  morphing  from  a 
circle  to  a  square,  a  circle  to  a  star,  and  a  star  to  a  pentagon  are  given  in  Figure  8-3. 


(a)  A  star  morphing  to  a  pentagon 


(b)  A  square  morphing  to  a  circle 


(c  )  A  circle  morphing  to  a  star 

Figure  8-3.  Examples  of  the  automatic  feature-based  morphing. 


8.4.  Future  Extension 

Although  we  have  made  the  assumptions  that  the  shapes  are  polygons  and  the  width  is  one  pixel, 
these  assumptions  can  be  easily  removed  by  some  modifications  of  the  algorithm.  A  proposed  approach  is 
to  find  the  ordered  pairs  of  points  on  a  scan  line  according  to  their  radial  distances  (which  is  the  distance 
to  the  center)  and  to  map  open  curves  to  open/closed  curves  by  considering  radial  angles. 
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9.  Conclusions 


In  this  study,  previous  work  on  uncertainty  visualization  has  been  reviewed.  Three  display 
techniques  were  selected  from  three  categories  for  the  comparison  study.  Scenarios  on  a  simulated  radar 
display  were  designed  to  test  subjects’  understanding  of  uncertainty  and  investigate  how  the  display 
techniques  affect  decision  making. 

A  toolkit  was  developed  to  display  the  symbols,  create  the  scenarios  and  collect  experimental 
data.  The  toolkit  also  provides  a  tool  for  experimenters  to  test  different  scenario  settings.  It  is  object- 
oriented,  and  extensible.  It  can  be  reused  in  the  future  for  studying  a  wide  range  of  symbol  types, 
constructing  other  scenarios  and  carrying  out  further  comparison  study  of  display  techniques. 

The  experiment  containing  six  visits  for  each  subject  was  done  at  North  Carolina  A&T  State 
Univ.  Subjects  were  recruited  from  undergraduate  and  graduate  students.  Eighteen  students  finished  all 
visits.  Five  scenarios  of  dynamically  displayed  symbols  had  been  used  in  the  experiment  to  compare  three 
display  techniques:  rectangular  bar,  color  saturation  and  blurred  image.  The  results  showed  that  display 
methods  affect  understanding  uncertainty  and  decision-making  performance  in  dynamic  display.  In  fixed 
location  problems  (scenarios  1  and  3),  the  results  indicate  that  there  is  significant  difference  on 
uncertainty  estimation  accuracy  related  to  the  selected  display  methods,  and  rectangular  bar  gives  the  best 
results.  In  translation  problems  (scenarios  2  and  4),  display  methods  affect  the  perception  of  the 
uncertainty  level,  but  not  the  location  information.  In  a  problem  with  multiple  symbols  whose  location 
and  state  change  continuously  (scenario  5),  color  saturation  gives  the  best  results.  The  difference  of 
workload  of  the  three  methods  is  not  significant.  In  all  scenarios,  experts  outperform  novices.  Thus  it 
indicates  that  understanding  the  dynamically  displayed  symbols  using  the  selected  techniques  can  be 
improved  by  training  and  practice.  On  the  other  hand,  it  shows  that  none  of  the  three  selected  display 
methods  are  self-explained,  and  their  association  to  the  uncertainty  is  not  intuitive. 

Feature-based  morphing  is  a  potential  method  for  displaying  uncertainty.  An  automatic  feature- 
based  morphing  algorithm  was  developed  for  polygon  shapes  with  one  pixel  width.  This  algorithm  can  be 
extended  to  remove  the  constraints.  The  implementation  of  this  algorithm  in  the  toolkit  enables  exploring 
this  method  in  the  future. 
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Appendix  A:  Toolkit  User  Manual 


To  run  the  program,  copy  the  Uncertainty  Vis  VI.  0.3  folder  to  the  directory  in  which  you  want  to 
place  it.  Open  the  subfolder  /release  and  double  click  on  UncertaintyVisVl.0. 3.exe.  The  data  are  saved  in 
OutputData.txt  in  the  same  directory.  So  please  make  sure  this  directory  is  writable. 


1.  User  Interface 

The  main  window  contains  a  menu  bar,  a  tool  bar,  two  dock  widgets,  a  status  bar,  and  a  display 

area. 

The  menu  bar  contains  the  following  items: 


File 

Exit 


Options 

Mode 

Customize  Scenario 

(Load  scenario  parameters  from  a  text  fde. 

See  the  experiment  plan  for  the  description  of  scenarios) 

Background 

Radar  Display 

White  Background 

Background  Image  (*.jpeg,  *.png) 

Help 

Display  Symbols 
Clear  Symbols 


By  default,  the  left  dock  widget  is  the  scenario  setting  dialog  box,  and  the  right  dock  widget  is  the 
user  input  dialog  box.  The  two  dock  widgets  can  be  floated  or  moved  to  the  right  or  left  dock  area  in  the 
main  window.  They  can  not  be  closed  or  resized.  The  widgets  in  user  input  dialog  box  will  change  when 
the  scenario  setting  changes. 

The  toolbar  contains  two  buttons:  play  and  stop  for  playing  the  selected  scenario. 

The  status  bar  shows  the  mode  in  which  the  program  is  running.  It  displays  either  experimenter 
mode  or  subject  mode. 


Keyboard  control: 
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The  two  groups  of  radio  buttons  (Changed  and  Unehanged  for  scenarios  1  and  3,  Friendly  and 
hostile  for  scenario  5)  have  keyboard  shortcuts: 


Changed:  c 
Unchanged:  u 
Friendly:  f 
Flostile:  h 


Cursors: 

Arrow:  used  for  accessing  menu  items,  input  widgets  such  as  combo  boxes,  spin  boxes  and 

sliders 

Cross:  used  for  specifying  points  in  the  drawing  area  (in  scenario  2  and  4).  Once  the  mouse  is 
clicked,  a  point  location  will  be  marked  with  a  red  square.  Multiple  points  are  connected  by  lines. 


Symbols: 

Friendly  and  hostile  symbols  are  shown  below. 


friendly  hostile 


2.  Running  the  Program 

2. 1  Running  the  program  as  an  experimenter 

Experimenters  can  change  parameters  for  all  scenarios  by  loading  a  parameter  file  from  the  menu 
Options>Customize  Scenario.  The  file  format  is  described  in  section  3. 

Under  the  experimenter  mode,  data  will  not  be  forced  to  save  before  a  new  scenario  is  played. 

2.2  Running  the  program  as  a  subject 

Subjects  need  to  run  the  program  following  the  experimenters’  instructions  on  which  type  of 
background  and  display  technique  to  select  and  in  which  order  should  the  scenarios  be  played.  Parameters 
can  be  loaded  from  a  parameter  file  from  the  menu  Options>Customize  Scenario. 

Under  the  subject  mode,  data  will  be  forced  to  save  before  a  new  scenario  is  played.  A  warning 
message  box  will  display  the  information  to  subjects. 


3.  Parameter  File  Format 

Loading  scenario  parameters  from  a  text  file  provides  the  convenience  for  experimenters  to 
explore  different  parameter  settings.  A  parameter  file  has  an  extension  *.txt.  To  load  a  parameter  file,  go 
to  the  menu  item  Options>Customize  Scenario,  and  select  the  file  name  in  the  file  open  dialog  box. 

The  parameters  are  loaded  automatically  from  “parameters.txt”  when  the  program  is  initialized. 
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The  origins  of  the  eoordinates  are  in  the  center  of  the  background.  For  the  radar  screen  and  white 
background,  the  width  is  800  and  the  height  is  600.  The  origin  of  a  symbol  is  at  its  left  upper  comer. 


A  line  by  line  explanation  of  the  file: 

Line  1:  Scenario  1  comments 

Line  2:  X,  Y,  StartingProbability,  EndProbability,  StartingAffiliation,  EndAffiliation,  Duration, 
NumFrames 

Fine  3:  Scenario2  comments 
Eine  4:  Duration,NumPoints 
Eine  5 :  X,  Y,  Probability 
Eine  6: 

Eine  (5+NumPoints  defined  in  scenario  2):  Scenarios  comments 

Eine  6+NumPoints:  X,  Y,  StartingSizeFactor,  EndSizeF actor,  StartingProbability,  EndProbability, 

Duration,  NumFrames 

Eine  7+NumPoints:  Scenario4  comments 

Eine  8+NumPoints:  X,  Y,  DeltaX,  DeltaY,  Duration,  NumSteps 

Eine  9+NumPoints:  Scenarios  comments 

Eine  1 0+NiimPoints:  NumSymbols,  Affiliation,  Radius,  MaxDistance,  Duration,  NumFrames,  NumPoints 
Eine  1 1+NumPoints:  X,  Y 

Eine  1 1+NumPoints  defined  in  scenario  2  +  NumPoints  defined  in  scenario  5 


Parameter  ranges: 

Probability  (double):  [0,  1.0] 

Affiliation  (integer)  :  0  friendly,  1  hostile 
Duration  (double): 
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In  scenario  1  &  3,  it  is  the  period  between  a  starting  symbol  and  an  end  symbol  >0.0 
In  scenario  2,  4  &  5,  it  is  the  period  that  a  symbol  stays  at  one  location. 

NumFrames  (integer)  total  number  of  frames  between  a  starting  symbol  and  an  ending  symbol  >1 
SizeF actor  (double)  :  >0  ( (0,1)  diminished  symbol,  (1, infinity)  enlarged  symbol) 

NumPoints  (integer):  >=1 

DeltaX(integer):  distance  in  x  direction  for  each  step  (see  the  explanation  of  scenario  4  below) 
DeltaY(integer):  distance  in  y  direction  for  each  step 
NumSteps  (integer):  >=1 
NumSymbols  (integer):  >=1 

Radius  (integer):  a  parameter  to  control  the  symbol  locations  (see  the  explanation  of  scenario  5) 
MaxDistance  (integer):  a  parameter  to  control  the  symbol  probability 

X  and  Y :  in  order  to  display  a  symbol  in  the  simulated  radar  screen  or  white  background,  X  should  be  a 
value  in  (-400,400),  Y  should  be  a  value  in  (-300,  300) 


In  scenario  4,  the  symbol’s  traveling  path  is  a  line  starting  from  x  and  y.  It  moves  DeltaX  and 
DeltaY  at  each  step.  When  displayed  to  subjects,  the  symbols  are  placed  at  a  random  place  around  the  real 
location.  The  maximum  distance  from  the  real  location  is  20  (pixels).  When  it  is  close  to  its  real  location, 
the  probability  level  is  high,  and  when  it  is  far  away  from  its  location,  the  probability  is  low.  Probability 
zero  corresponds  to  a  distance  of  20  pixels  and  probability  one  corresponds  to  a  distance  of  zero.  Subjects 
are  asked  to  guess  the  next  move. 

In  scenario  5,  several  symbols,  which  represent  the  target  group,  undergo  a  fixed  movement 
toward  the  observer.  When  the  target  is  far  away  from  the  observer,  the  probability  of  getting  accurate 
information  is  low.  As  the  target  moves  toward  the  observer,  the  information  collected  is  more  accurate. 
The  probability  is  a  function  of  the  distance.  P  =  1.0  -  Jzxt/ MaxDistance  ,  where  dist  is  the  current 
distance  between  the  target  and  the  observer,  and  MaxDistance  is  the  distance  at  which  we  assume  the 
probability  equals  0.  The  target  group  can  be  friendly  or  hostile.  A  symbol  will  be  set  by  the  probability. 
Assume  that  the  target  is  hostile.  A  random  number  r  (between  0  and  1)  will  be  generated  for  each 
symbol.  If  r>P,  the  symbol  is  set  as  friendly;  otherwise  it  will  be  set  to  hostile.  (x,y)  is  the  location  of 
the  group.  Symbols  are  located  randomly  in  a  circular  area  centered  at  (x,y).  The  MaxDistance  and 
Radius  can  be  set  in  the  parameter  file.  Cross  dissolve  is  used  for  changing  state  from  one  to  the  other. 


4.  Output  File  Format 

Subjects’  responses  to  each  scenario  are  saved  in  “OutputData.txt”  in  the  same  directory  of  the 
executable.  An  example  of  this  file  is  shown  below. 

Each  time  a  scenario  is  played,  a  line  describing  the  scenario  name  and  the  display  technique  is 
recorded.  Each  time  the  save  button  in  the  user  input  dialog  box  is  clicked,  the  data  are  saved. 

Scenario  1,  Rectangular  Bar 
record: 

Time:  3 

Choice:  changed 

starting  probability  level:  0.5 
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end  probability  level:  0.2 


Scenario  2,  Rectangular  Bar 
record: 

Point:  1 

probability:  0.2 
xy:  128,177 
Point:  2 
probability:  0.4 
xy:  119,90 
Point:  3 

probability:  0.6 
xy:  79,37 
Point:  4 
probability:  1 
xy:  12,-22 

Scenario  3,  Rectangular  Bar 
record: 

Time:  1.8 
Choice:  changed 
starting  probability  level:  0.5 
end  probability  level:  0.9 

Scenario  4,  Blurred  Image 
record: 

Time:  8.9 
XY:  -37,  -49 


Scenario  5,  Blurred  Image 
record: 

Time:  21.6 
Choice:  hostile 


5.  Files  Included  in  this  Release 

README.doc 

README.pdf 

Release/ 

UncertainVisV  1 .0. 3.exe 

Parameters.txt 

Mingwml0.dll 

QtCore4.dll 

QtCored4.dll 
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QtGui4.dll 

QtGuid4.dll 

Images/ 

AircraftFriendly.png 

AircraftHostile.png 

Icon/ 

exit.png 

player_play.png 
play  erstop. png 
stop.png 
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List  of  Symbols 


Graphical  objects  0 

Geometric  data  set  U 
Set  of  ordered  pairs  P 
Family  of  transformation  W 
Source  set  S 
Target  set  D 
Point  (x,y) 
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